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Chernoff Bounds 


6.2.3 Chernoff Bounds 

If X is a random variable, then for any a G M, we can write 

P(X >a) = P(e sX > e sa ), for s > 0, 

P[X <a) = P{e sX > e sa ), for s < 0. 

Now, note that e sX is always a positive random variable for all S G M. Thus, we can apply 
Markov's inequality. So for S > 0, we can write 

P(X > a) = P(e sX > e sa ) 

E\e sX ] 

< -, by Markov’s inequality. 

e sa 

Similarly, for S < 0, we can write 

P(X < a) = P{e sX > e sa ) 



Since Chernoff bounds are valid for all values of S > 0 and S < 0, we can choose S in a 
way to obtain the best bound, that is we can write 
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P(X >a)< mme~ sa Mx(s), 

s> 0 

P(X < a) < min e~ sa M x (s). 

s<0 

Let us look at an example to see how we can use Chemoff bounds. 


Example 6.22 

Let X ~ Binomial (n, p). Using Chemoff bounds, find an upper bound on 
P(X > an), where P < a < 1 . Evaluate the bound for p — ^ and a — 

• Solution 

o For X ~ Binomial(n,p), we have 

Mx(s) — ( pe s + q) n , where q — 1 — p. 
Thus, the Chemoff bound for P[X > a) can be written as 
P(X > an) < min e~ sa Mx{s) 

s>0 

= mine _sa (pe s + q ) n . 

s>0 


To find the minimizing value of S, we can write 


d 


e ~ sa (pe s +q) n = 0 , 
as 


which results in 


e s = 


aq 


np( 1 — a) 

By using this value of s in Equation 6.3 and some algebra, we obtain 
P(X > an) < . 

1 3 

For p — — and a — -,we obtain 

P(X > -n) < ( 27 ) 4 ■ 
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Comparison between Markov, Chebyshev, and Chemoff Bounds: 

Above, we found upper bounds on P[X > cm) for X ~ Binomial(n, p ). It is 

1 3 

interesting to compare them. Here are the results that we obtain for p — and On. = : 


P(X > 
P(X > 
P(X > 



Markov, 

Chebyshev, 

Chernoff. 


The bound given by Markov is the "weakest" one. It is constant and does not change as 71 
increases. The bound given by Chebyshev's inequality is "stronger" than the one given by 

Markov's inequality. In particular, note that goes to zero as 71 goes to infinity. The strongest 
bound is the Chernoff bound. It goes to zero exponentially fast. 


<— previous 
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